Hi Boston!

• I’m Jasmine Dumas
  • Data Scientist and useR living in Connecticut
  • Work at Simple Finance based in Portland, OR
  • Author of ttbbeer and shinyLP
• Slides and Materials available on GitHub:
  • jasdumas/talks/odsc-boston

Why should you care open government data?

To foster and improve data literacy & statistical comprehension within our community, Scientists and Engineers need to advocate for data that is provided in an consistent and accessible method suitable for analysis.

What this presentation is about (and not about)

2017 has been an interesting time to be a Data Scientist…

https://www.nytimes.com/2017/03/06/science/donald-trump-data-rescue-science.html?smid=tw-nytimesscience&smtyp=cur&_r=0

What even is, beer analytics?

• The use of quantitative methods to:
  • improve the production & quality of beer
  • measure customer preferences and predict seasonality
  • gather insights on industry performance

Discovering analysis-ready beer datasets can be difficult

How to generally search for analysis-ready datasets

The U.S. Government Open Data Portal

The “clearinghouse” for open U.S. government data is located at data.gov. It also contains tools & resources to conduct research, develop web and mobile applications, and design data visualizations.

Examples of analysis-ready datasets

• Analysis-ready datasets can be obtained with R functions like:
  • read.csv() or read.table()
• “Great” analysis-ready datasets take minimal cleaning and have interpretable data fields
• Analysis-ready data also needs to include some type of documentation such as a data dictionary

Examples of datasets that are not analysis-ready

• Datasets that have inconsistent formats, mixed column types, and cryptic data fields make it difficult for analysis (not tidy)

Here is the point of all of this…

Just because it’s open doesn’t mean it’s accessible!

Necessity breeds innovation

• I was inspired to become better at finding datasets and web scraping from the road-blocks of finding analysis-ready datasets for projects about beer!

• Here is a tutorial I wrote to get meta data on brewery ratings in Connecticut from Beer Advocate using Import.io and processed the data with R.

I developed a R package for beer statistics, called ttbbeer

How I developed the ttbbeer v1.0.0 package

• Hand transcribing from PDF documents to an excel workbook 😢

##      Month Year Malt_and_malt_products Corn_and._corn_products
## 1  January 2015              316756669                60186876
## 2 February 2015              294630835                49718568
## 3    March 2015              320802302                53925263
##   Rice_and_rice_products Barley_and_barley_products
## 1               52438465                    8492680
## 2               48975137                    9355462
## 3               48710754                   22213428
##   Wheat_and_wheat_products Sugar_and_syrups Hops_dry Hops_extracts
## 1                  1392183         72069615  1916516        298514
## 2                  2145359         74431855  1775144        298894
## 3                  3761816         82679762  3750830        321462
##      Other
## 1 11163548
## 2 12017223
## 3 13892190

How I developed the ttbbeer v1.1.0 package

• Web-scraping using the R package rvest 😄
• Data cleaning & parsing date mistakes with stringr: "101/01/1951"

Insights from this data

Advocating for open data

https://www.data.gov/issues/request-id/32022/

Wrapping things up

Questions & Discussion!